fix: preserve picklescan stack state by mldangelo-oai · Pull Request #910 · promptfoo/modelaudit

mldangelo-oai · 2026-04-10T17:41:17Z

Summary

Preserve standalone picklescan symbolic stack state for DUP, dict/list/set mutation opcodes, and explicit memo writes before MEMOIZE.
Add regressions for SETITEM/SETITEMS operands so nested dangerous globals do not get misattributed as REDUCE call targets.
Add a memo-index regression so MEMOIZE after PUT/BINPUT/LONG_BINPUT resolves later BINGET call targets correctly.

Validation

uv run pytest packages/modelaudit-picklescan/tests
uv run ruff format modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
uv run ruff check --fix modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
uv run mypy modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
uv run pytest -n auto -m "not slow and not integration" --maxfail=1
uv run ruff format --check modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
uv run ruff check modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
git diff --check

Summary by CodeRabbit

Bug Fixes
- Enhanced pickle scanning reliability by preserving stack state across memo and container mutation opcodes, preventing state loss during analysis of complex pickle files
Tests
- Added test coverage validating that dictionary mutation patterns and memoization-index behaviors are properly detected as threatening operations during pickle scanning

coderabbitai · 2026-04-10T17:41:23Z

Warning

Rate limit exceeded

@mldangelo-oai has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 14 minutes and 7 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 14 minutes and 7 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7399e4e7-3018-49cf-84d5-e3ea407d6333

📥 Commits

Reviewing files that changed from the base of the PR and between fd935af and dedfd0a.

📒 Files selected for processing (3)

CHANGELOG.md
packages/modelaudit-picklescan/src/modelaudit_picklescan/engine/scanner.py
packages/modelaudit-picklescan/tests/test_api.py

Walkthrough

This pull request updates the pickle scanner's state management and opcode handling. Changes include resetting memo state only on STOP, implementing explicit stack duplication for DUP, refining memo-write semantics for opcodes like APPEND and SETITEM, and replacing indexed memo tracking with length-based keys. Test coverage is added for detecting dangerous pickle patterns.

Changes

Cohort / File(s)	Summary
Documentation `CHANGELOG.md`	Added bug fix entry documenting preservation of picklescan stack state across memo and container mutation opcodes.
Scanner Logic `packages/modelaudit-picklescan/src/modelaudit_picklescan/engine/scanner.py`	Refactored memo state management to reset only on `STOP` opcode, added explicit `DUP` stack duplication, altered memo-write semantics for `APPEND`/`SETITEM`/bulk variants, and changed `MEMOIZE` to use `len(self.memo)` instead of separate index tracking.
Test Coverage `packages/modelaudit-picklescan/tests/test_api.py`	Added two parametrized test cases validating detection of dangerous behavior in dictionary mutation and memoization-index patterns in pickled objects.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🥕 Our scanner hops with sharper sight,
Memo states aligned just right,
Stack and opcodes dance in flow,
Dangerous pickles won't elude, no!
Tests now catch what once slipped by—
Security soars! Bunny says "Hi!" 🐰

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: preserve picklescan stack state' directly corresponds to the main objective and changes in the PR, which focus on preserving stack state across opcodes and memo handling.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch mdangelo/codex/picklescan-audit-pass-2

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 916cfaf856

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

packages/modelaudit-picklescan/src/modelaudit_picklescan/engine/scanner.py

…scan-audit-pass-2 # Conflicts: # CHANGELOG.md

fix: preserve picklescan stack state

916cfaf

mldangelo-oai enabled auto-merge (squash) April 10, 2026 17:41

chatgpt-codex-connector bot reviewed Apr 10, 2026

View reviewed changes

packages/modelaudit-picklescan/src/modelaudit_picklescan/engine/scanner.py Outdated Show resolved Hide resolved

mldangelo-oai added 2 commits April 10, 2026 11:00

fix: use memo size for picklescan memoize

fd935af

Merge remote-tracking branch 'origin/main' into mdangelo/codex/pickle…

dedfd0a

…scan-audit-pass-2 # Conflicts: # CHANGELOG.md

mldangelo-oai merged commit fabac5c into main Apr 11, 2026
22 checks passed

mldangelo-oai deleted the mdangelo/codex/picklescan-audit-pass-2 branch April 11, 2026 05:42

github-actions bot mentioned this pull request Apr 11, 2026

chore(main): release 0.2.35 #929

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: preserve picklescan stack state#910

fix: preserve picklescan stack state#910
mldangelo-oai merged 3 commits intomainfrom
mdangelo/codex/picklescan-audit-pass-2

mldangelo-oai commented Apr 10, 2026 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Apr 10, 2026 •

edited

Loading

Rate limit exceeded

❌ Failed checks (1 warning)

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

mldangelo-oai commented Apr 10, 2026 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rate limit exceeded

Walkthrough

Changes

Estimated code review effort

Poem

❌ Failed checks (1 warning)

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mldangelo-oai commented Apr 10, 2026 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Apr 10, 2026 •

edited

Loading